Rate control is a component that performs a critical function in a modern video encoder. It ensures that the generated compressed bit stream (a) achieves the bit rate target, (b) satisfies maximum average bit rate constraints, and (c) satisfies hypothetical reference decoder (buffering delay) constraints, among others. An optional, but highly desirable, objective is to optimize the video quality of the compressed video bitstream.
Satisfying constraint (a) ensures that the compressed video bitstream will fit the communication pipe or storage space. For example, a wireless network may only allow up to 768 kbps for video transmission. This is in contrast to, say, a Blu-Ray disc, which may allow up to 40 Mbps of video bandwidth for traditional 2D applications and 60 Mbps of bandwidth for 3D applications employing codecs based on the H.264/MPEG-4 AVC video coding standard [see reference 1, incorporated herein by reference in its entirety] and its extensions, i.e. the Multi-view Video Coding (MVC) extension of MPEG-4 AVC. In addition, for archival applications or applications where bandwidth can be extremely high (such as reading from a hard drive) one may only specify the total size of the final bitstream file.
Constraint (b) derives from limitations of playback devices, which can only store and decode a certain number of bits per second. It is possible that during the encoding process a situation may occur where the average bit rate for the entire compressed bitstream achieves the bit rate target, but the average bit rate exceeds it locally, for example for a duration of some seconds. Often this can happen because difficult to code areas usually require more bits to ensure consistent or better video quality. However, these bitstream “spikes”, provided they are large enough, can create problems for resource-constrained decoders. Problems that may affect a decoder include either overflowing of internal buffers or the inability to decode the bitstream in time to display the frames in the correct order and with proper timing.
Last, constraint (c) is closely related to constraint (b) and can be thought of as a more rigorous set of requirements that a bitstream has to satisfy. In short, the compressed bitstream has to be coded such that if transmitted at the target bit rate it will never cause a decoder buffer overflow or underflow, and as a result, the decoded video will never stall or stop during playback.
Rate control is also tasked with ensuring the best possible video quality given the above bit rate and buffering constraints.